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The  possibility  is  considered  that  research  on  intelligence  is  enter¬ 
ing  or  is  about  to  enter  a  time  of  crisis  comparable  to  that  experienced 
during  the  decline  of  the  psychometric  paradigm  as  the  primary  means  for 
studying  intelligence.  First,  it  is  suggested  that  the  decline  of  t fie 
psychometric  paradigm  as  the  primary  means  for  studying  intelligence  was 
due  in  part  to  the  failure  of  users  of  the  paradigm  to  meet  in  a  highly 
successful  way  four  challenges  that  confronted  their  research.  Next, 
it  is  shown  how,  on  the  surface,  users  of  the  information-processing 
paradigms  currently  in  favor  seem  successfully  to  have  met  these  chal¬ 
lenges.  Then,  it  is  shown  that,  at  a  deeper  level,  the  level  of  success 
is  not  as  great  as  it  is  at  a  surface  level.  Finally,  conclusions  are 
drawn  in  response  to  the  challenges  that  still  seem  to  be  facing  psy¬ 


chologists  studying  intelligence. 
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Nothing  Tails  Like  Success: 

The  Search  for  an  Intelligent  Paradigm  for  Studying  Intelligence 

I  believe  the  time  has  come  at  least  to  consider  the  possibility  that 
research  on  intelligence  is  entering  or  is  about  to  enter  a  time  of  crisis 
and  soul-searching  comparable  to  that  experienced  during  the  nineteen 
fifties  and  sixties,  when  researchers  experienced  certain  dissatisfactions 
with  the  psychometric  paradigm  as  the  sole  or  primary  means  for  studying 
intelligence,  but  were  not  quite  sure  of  what  should  replace  or  supplement 
it.  My  contention  is  that  at  one  level— the  level  that  meets  the  eye  upon 
a  superficial  examination  of  the  present  condition  of  intelligence  research- 
current  paradigms  for  studying  intelligence  have  successfully  faced  the 
problems  that  factor  analysis  seemed  to  face  in  a  less  than  wholly  successful 
way,  but  that  at  another  level --the  level  that  meets  the  eye  upon  a  deeper 
examination  of  the  present  condition  of  intelligence  research— current 
paradigms  are  not  facing  these  problems  altogether  successfully. 

My  exposition  is  divided  into  three  parts.  In  the  first,  I  state  what 
I  believe  to  have  been  four  of  the  major  challenges  that  the  psychometric 
paradigm  for  studying  intelligence,  in  general,  and  the  factor-analytic 
approach,  in  particular,  failed  fully  to  meet.  In  the  second  part,  I  first  s 
why,  at  a  superficial  level,  at  least,  current  approaches  based  upon  the 
information-processing  analysis  of  intelligent  behavior  are  seeming  to 
meet  these  challenges;  I  then  show  why,  at.  a  deeper  level,  1  believe  all  of 
these  challenges  have  yet  to  be  confronted  head  on.  In  the  third  and  final 
part,  I  discuss  what  might  be  done  to  meet  these  challenges.  Because  of  the 
seemingly  negative  tone  of  many  earlier  parts  of  the  article,  I  wish  to  erpha 
size  here  that  these  conclusion,  will  be  opt  i ,ii st ic-- tha t  although  . a 1 1  Inown 
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result  in  major  advances  in  our  knowledge  concerning  the  nature  of  intelligence. 

The  Four  Challenges  and  Factor  Analysis 

Factor  analysis  has  been  anc  continues  to  be  a  highly  useful  tool  for 
studying  intellectual  functioning;  Nothing  1  will  say  is  able  or  intended 
to  refute  this  assertion.  But  factor  analysis,  like  any  other  method  of 
data  analysis,  is  unable  to  go  it.  alone.  I  believe  that  in  the  case  of 
factor  analysis,  there  are  four  reasons  why  supplementary  methods  of 
analysis  are  particularly  important  in  the  study  of  intelligence. 

First,  as  Humphreys  pointed  out  almost  two  decades  ago,  factor  analy¬ 
sis  is  a  "useful  tool  in  hypothesis  formation  rather  than  hypothesis  testing" 
(Humphreys,  1962,  p.  475).  Factor  analysis  is  useful  in  hypothesis  forma¬ 
tion  because  one  can  go  into  it  with  few  or  even  no  ideas  about  the  struc¬ 
ture  underlying  a  set  of  variables,  and  come  out  of  it  with  at.  least  some 
idea  of  what  this  latent  structure  looks  like.  I  believe  that  noncon f i rmatory 
factor  analysis  is  not  useful  in  hypothesis  testing,  however,  because  the  in¬ 
ferential  machinery  supporting  it  is  so  weak.1  There  have  been,  of  coui so, 
prominent  investigators  who  have  taken  and  still  would  take  issue  with  this 
point  of  view.  Burt  (despite  his  apparent  proclivity  toward  "assisting"  his 
data,  a  competent  factor  analyst  nevertheless)  argued  that  factor  analysis  should 
be  regarded  "not  as  a  source  of  hypotheses,  but  merely  as  a  method  of  comparing, 
confirming  or  refuting  alternative  hypotheses  initially  suggested  by  nonsta- 
tistical  arguments  or  evidence"  (hurt,  1970,  p.  17).  More  recently,  Carroll 
(in  press)  has  argued  that  "the  machinery  of  factor  analysis  need  not  U 
dependent  on  any  hypotheses  adopted  in  advance  of  the  analysis;  c-illy 
it  affords  a  way  of  testing  those  hypotheses,"  a  way  that  Carroll  'eel  i  -ves 
to  be  "appropriate  and  sufficiently  objective."  But  I  cannot  thiuk  single 

plausible  psychological  hypothesis  whose  validity  has  been  i  or/,  lu.ic  lv 
tested  and  established  (or  disestablished)  thuuigh  the  use  of 


(nonconfi rmatory)  factorial  methods,  despite  the  fact  that  these  methods 
have  been  around  for  three-quarters  of  a  century.  Consider,  for  example, 
the  very  basic  question  of  whether  there  is  a  general  factor  in  intelli¬ 
gence.  Almost  eighty  years  after  the  first  presentation  of  Spearman's 
(1904)  two-factor  theory,  has  anyone  answered  through  factorial  means 
the  question  of  whether  or  not  a  general  factor  exists?  The  contents  of 
a  set  of  16  commentaries  on  an  article  I  recently  wrote  (Sternberg,  in 
press-c)  make  it  clear  that  no  one  has:  Investigators  disagree  as  much  now 
as  they  did  at  the  turn  of  the  century  as  to  whether  to  interpret  factorial 
evidence  as  supporting  or  refuting  the  existence  of  a  general  factor. 

Because  of  the  weakness  of  its  inferential  machinery,  factor  analysis 
has,  in  a  sense,  failed  because  it  has  been  too  successful  in  supporting,  or 
at  least  in  failing  to  disconfirm,  too  many  alternative  models  of  intelligence 
Horn  (1967;  Horn  &  Knapp,  1973)  has  suggested  that  Guilford's  (1967)  theory,  it 
particular,  is  exceedingly  difficult  to  disconfirm  because  of  tiie  way  in  which 
Procrustean  rotation  is  used.  I  have  gone  further  in  suggesting  that 
none  of  the  factorial  theories  are  di sconfirmabl e ,  because  in  major 
respects,  all  of  them  are  correct  (Sternberg,  1980a  )!  They  highlight 

di fferent  aspects  of  intelligence,  all  of  which  can  be  mapped  into  informa¬ 
tion-processing  terms.  On  the  positive  side,  therefore,  the  theories 
have  provided  a  richly  variegated  account  of  human  intelligence;  on  the 
negative  side,  however,  I  do  not  see  how  intelligent  use  of  factorial 
methods  can  fail  to  yield  a  legitimate  theory.  But  science  in  general, 
and  psychology  in  particular,  progress  at  least  in  part  by  tentatively 
accepting  certain  accounts  of  phenomena ,  and  by  tentatively  reject,  iv.  <  /  r  • ; 
factor  analysis  has  left  us  little,  if  anything,  to  reject.  Wn.it  plu;.iM< 
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theories  of  intelligence  have  been  discontinued  by  factor  analysis?  My 
contention  is  that  not  only  are  the  major  alternative  factorial  theories 
mathematically  tenable;  they  are  psycho! ogically  tenable  as  well.  The 
method  might  therefore  be  viewed  as  having  failed,  in  practice,  because 
it  has  been  too  successful! 

Second,  factor  analysis  has  never  seemed  to  be  a  technique  of  choice 
if  one's  goal  is  to  identify  the  processes  that  constitute  intelligent  be¬ 
havior.  Factor  analysis  has  dealt  primarily  with  products  rather  than 
with  processes.  Even  in  recent  work  using  confirmatory  methods  (e.g., 

Frederiksen,  1980),  identification  of  processes  has  been  through  standard 
information-processing  techniques,  such  as  the  subtraction  method  and 
the  additive-factor  method.  Confirmatory  analysis,  e.g.,  analysis  of 
covariance  structures  (Joreskog,  1970),  has  been  used  to  isolate  common 
sources  of  individual  differences  in  execution  of  these  processes  (which  could  in¬ 
clude  as  well  as  common  processes,  common  representations  of  information, 
common  input  modalities,  common  psychological  units  of  analysis,  etc.). 

Perhaps  potentially,  factor  analysis  might  have  told  us  or  still  might 
tell  us  more  about  information  processing  than  it  has.  The  past  evidence, 
however,  suggests  that  it  is  not  a  useful  technique  for  separating  process 
from  other  sources  of  individual  differences,  such  as  content.  In  Thurstons 1 s 
(1938)  theory,  for  example,  the  distinction  between  process  and  content  is 
not  clear.  Tho  one  theory  in  which  the  distinction  is  very  cleai  i.> 

Guilford's,  but  I  suspect  this  reflects  more  Guilford's  creative  concc p;ual i - 
zation  of  intelligence  than  the  results  of  factor  analyses  perform.!  on 
Guilford’s  or  others'  data  (see  the  papers  by  Horn  cited  earlier).  The 
fact  of  tho  matter  is  that  at  least  until  now,  factor  analysis  ha:,  not  been 


at  its  best  in  elucidating  the  processes  constituting  intelligent  ; 


■i  i  or  a iv.  <• . 


Third,  by  the  end  of  tho  sixties,  it  seemed  as  though  noncnnfn  •lory 
had  told  us  pf-tty  mucli  wh.it  i  !  Was  going  to  I'  ll  us  about  tho  n  m;,  ..  j.  . 


v-;  sf  *e  o*. 
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telligonce.  This  view  is  not  a  negative  assessment  of  the  cumulative  con¬ 
tribution  of  factor  analysis;  rather,  it  represents  a  belief  that  after 
three-quarters  of  a  century,  nonconfi rmatory  versions  of  the  technique, 
at  least  as  they  have  been  used  in  the  past  for  the  analysis  of  IQ-test 
items,  have  pretty  much  been  milked  dry.  Carroll  (  1980  )  has  suggested 

that 

factor  analysis  is  not  at  all  as  'indeterminate'  as  it  is  often 
depicted  to  be,  and  as  it  was  in  fact  depicted  by  Sternberg  in 
his  book  (1977).  With  wel 1 -desi gned  studies,  the  principles 
of  simple  structure  can  pretty  well  dictate  the  final  solution. 

Parsimony  is  the  essential  principle  underlying  the  idea  of 
simple  structure;  it  says  that  one  wants  to  account  for  a  given 
variable  with  the  minimal  number  of  factors-~often  with  only 
one  factor. 

I  do  not  agree  with  Carroll  that  the  issue  of  a  preferred  rotation  is  dic¬ 
tated  by  anything,  other  than  the  mathematical  constraints  of  rotation, 
which  allow  an  infinity  of  valid  rotations  rather  than  a  single  one.  I 
also  consider  parsimony  to  be  only  one  of  a  number  of  criteria  one  should 
use  in  assessing  the  value  of  a  factorial  theory,  or  any  other  kind  of 
theory,  for  that  matter. (See  Sternberg,  1977,  Chapter  5,  for  a  discussion 
of  some  other  criteria--completeness ,  specificity,  generality,  plausibility-- 
that  matter  as  well.)  And  certainly,  the  issue  of  a  preferred  rotation 
(like,  it  seems,  many  other  issues  pertaining  to  factor  analysis)  regains  a 
matter  of  debate.  Contemporary  factor  theorists  other  than  Carroll  (o.g., 
Cattell,  1971;  Guilford,  1967;  Horn,  1968;  Snow,  1978;  Vernon,  1971;  to 
name  a  few)  continue  to  use  solutions  other  than  simple-structure  ones  in 
their  theorizing  about  intelligence.  But  if  the  issue  of  a  "corro.  ■"  ro¬ 
tation  had  been  decided,  then  I  would  be  even  r.nrc-  convinced  th.t  tr.-di- 


■tk* 
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tional  factorial  techniques  have  not  recently  been  telling  us  a  great  deal 
new  about  the  nature  of  intelligence:  I  don't  believe  we’ve  learned  a 
great  deal  more  about  the  simple-structure  nature  of  intelligence  than 
we  knew  from  Thurstone's  early  investigations  of  it  (e.g.,  Thurstone,  193C) 
I  am  not  stating  a  belief  that  simple-structure  solutions  do  not  provide 
us  with  useful  information  about  intelligence.  To  the  contrary,  I  believe 
that  these  solutions,  and  others  as  well,  provide  us  with  a  great  deal  of 
useful  information  (see  Sternberg,  1980a  »  in  press-c).  But  1  do  not 

see  that  with  the  coming  of  the  nineteen  eighties,  they  are  likely  to  pro¬ 
vide  us  with  much  new  information,  unless  they  are  applied  to  new  materials 
or  otherwise  applied  in  new  ways. 

Fourth  and  finally,  I  believe  that  by  the  end  of  the  nineteer  sixties, 
factor-analytically  derived  theories  were  being  perceived  as  less  infor.-a- 
tive  than  might  have  been  hoped  with  regard  to  their  implications  for  instr 
tion,  in  particular,  instruction  in  intelligent  information  processing.  Pr 
sumably,  these  theories  were  less  helpful  in  this  regard  than  one  might 
have  hoped  because  they  did  not  make  clear  just,  what  it  is  that  should  be 
trained.  Obviously,  one  could  train  subjects  in  their  performance  on  the 
items  that  compose  the  factors.  But  although  the  theories  may  delimit 
the  class  of  items  to  be  trained,  they  do  not  indicate  how  to  train  per¬ 
formance  on  these  items.  Instead  of  decomposing  the  items  into  S’  oiler, 

!  more  concrete  and  potentially  trainable  units,  they  relate  the  it.  :r,  to 
factors  that  are  larger,  more  abstract,  and  probably  less  train..!  1  -  than 
the  items.  It  is  easier  to  see,. for  example,  how  one  might  train  analogy 
performance  than  it  is  to  see  how  one  might  Ira  in  performance  or.  a  f.n  tor 
of  "reasoning"  or  of  "general  intelligence."  One  possible  excp'i.'n  u> 
this  generalization  (and  there  may  well  be  others)  is  Guilford's  ( 1 ' :  ■  7 ) 
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theory,  where  the  factors  seem  to  specify  fairly  elementary  processes,  con¬ 
tents,  and  products.  Here,  oddly  enough,  the  problem  might  not  be  in  the 
paucity  of  implications  for  training,  but  in  their  plethora:  There  are 
120  abilities  postulated  in  the  model. 

The  Four  Challenges  e_nd_  1 nformati on-processing  Analysis 
An  Opt i mi sti c  View 

At  first  inspection,  contemporary  information-processing  approaches 
to  intelligence  seem  to  be  doing  admirably  in  meeting  squarely  the  chal¬ 
lenges  factor  analysis  met  less  than  adequately.  Indeed,  contemporary 
approaches  to  the  study  of  intelligence  were  formulated  at  least  in  part 
to  mitigate  these  and  other  apparent  "inadequacies"  of  factor  analysis 
used  in  the  absence  of  converging  methods  of  analysis. 

First,  the  methods  of  data  analysis  used  in  information-processing 
investigations  of  intelligence  are  highly  useful  in  hypothesis  testing 
(although  probably  less  useful  in  hypothesis  formation).  Although  the 
major  statistical  methods  used  in  such  investigati ons--analysi s  of  vari¬ 
ance  and  regression-~are  based  upon  the  same  general  linear  model  upon 
which  factor  analysis  is  based,  inferential  statistics  for  the  former 
two  methods  are  much  more  highly  developed  than  they  are  for  factor  analy¬ 
sis.  One  could,  of  course,  argue  over  the  potentials  for  hypothesis  testing 
of  factor  analysis  as  opposed  to  analysis  of  variance  or  multiple  regression 
But  to  a  large  extent,  the  "proof  of  the  pudding  is  in  the  tasting,"  and  eve 
a  perfunctory  review  of  the  two  literatures  will  reveal  a  much  greater  em¬ 
phasis  upon,  and  more  successful  use  of,  hypothesis  testing  in  the  informal!' 
processing  literature  than  in  the  factor-analytic  one.  Jn  one  kind  of 
information-processing  analysis,  computer  simulation,  the  use  of  inferential 
statistics  is  often  at  a  minimum.  But  even  hero,  strict  hypothesis  to:, linn 
is  possible,  albeit  hypothesis  testing  of  a  different  kind:  The  >  r  elator 
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can  test  whether  his  or  her  program  is  a  sufficient  account  of  behavior  si-:.;  1  , 
by  seeing  whether  the  program  (a)  runs  and  (b)  produces  the  desired  pattern 
of  responses.  And  these,  after  all,  are  the  tests  that  the  simulator  is 
interested  in  demonstrating  that  his  or  her  program  can  pass.  Reviews  and 
examples  of  the  kinds  of  inferential  tests  that  can  be  performed  in  the 
analysis  of  human-subject  data  collected  viu  the  information-processing 
approach  can  he  found  in  Sternberg  (  1  977  ,  1 07c'. ,  1980b). 

Second,  information-processing  techniques  such  as  the  subtraction 
method  and  the  additive-factor  method  (see  Parhelia,  1974,  for  a  readable 
description  of  these  two  methods)  are  highly  useful  in  identifying 
processes  that  contribute  to  intelligent  performance.  Indeed,  these  tech¬ 
niques  were  formulated  primarily  with  this  goal  in  mind.  It  is  often 
possible  not  only  to  identify  these  processes,  but  to  identify  as  well 
the  latencies  or  difficulties  of  the  processes,  the  strategies  into  which 
the  processes  combine,  the  representations  upon  which  the  processes  act, 
and  the  consistency  over  time  with  which  the  processes  are  used.  Examples 
of  such  analyses  can  be  found  in  the  work  of  Hunt  (Hunt,  Frost,  &  Lunne- 
borg,  1973;  Hunt,  Lunneborg,  &  Lewis,  1970),  Pellegrino  and  Glaser  (1930),  R. 
Sternberg  (1977  ,  1980b)  ,S.  Sternberg  (1  969),  and  many  others. 

Third,  I  do  not  sec  any  indications  that  wo  have  exhausted  the  poten¬ 
tial  of  the  information-processing  paradigm  to  yield  new  insights  into  the 
nature  of  intelligence,  nor  do  I  see  indications  from  the  literature  that 
others  see  this  as  an  impending  problem  either.  To  t lie*  contrary,  it  lies 


only  been  during  the  past  decade  that  information-processing  analy.es  of 
intelligent  behavior  have  made  a  serious  and  concerted  start  (with  a  t*-w 
earlier  exceptions;  set-,  e.g.  ,  Gagne,  1967),  and  all  indications  m  i  to 
be  that  it  will  be  quite  a  while  before  the  paradigm  is  exhausted  m  its 
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ability  to  yield  new  and  interesting  findings.  Obviously,  there  is  no 
"acid  test"  of  what  constitutes  a  sti 1 1 -productive  paradigm.  But  the  sheer 
volume  of  work  being  done  in  a  large  and  growing  number  of  different 
laboratories  would  seem  to  attest  to  the  productivity  of  the  paradigm  in 
generating  what  many  investigators  apparently  believe  to  be  worthwhile  re¬ 
search.  In  contrast,  by  the  late  nineteen  sixties,  the  proportion  of  researcher^ 
still  doing  factorial  analyses  of  intelligence  had  dwindled  to  a  low 
level,  after  a  prolonged  period  of  slow  but  steady  decline.  Some  of  the 
interesting  theoretical  pursuits  currently  in  progress  include  attempts 
to  account  for  intelligent  information  processing  in  a  variety  of  tasks 
via  a  relatively  small  number  of  information-processing  components  (c.g., 

Carroll,  1976;  Hunt,  1978;  Jensen,  1979;  Pellegrino  &  Glaser,  1 980 ;  Stern¬ 
berg,  1979);  attempts  to  build  computer  models  that  can  perform  a  wide  va¬ 
riety  of  tasks  intelligently  (e.g.,  Anderson,  1976;  Schank  &  Abel  son,  1  977  ; 

Simon  &  Lea,  1974);  and  attempts  to  account  for  mental  retardation  in 
information-processing  terms  (e.g.,  Butterfield  &  Belmont,  1977;  Campione 
&  Brown ,  1 97  8) . 

Fourth  and  finally,  information-processing  accounts  of  intelligent 
performance  seem  to  provide  implications  for  and  be  conducive  to  training 
because  they  decompose  tasks  into  component  processes  that  seem  to  be  (at 
least  in  some  cases)  of  a  level  of  complexity  that  is  compatible  with  in¬ 
structional  attempts.  Training  of  intelligent  performance  has  leaned  heavily 
and  directly  upon  the  information-processing  theories  from  which  the  training 
has  been  derived,  and  has  been  done  successfully  in  a  number  of  domains, 
including  learning  (e.g.,  Belmont  Butterfield,  1 971  ;  Camp i one  S  I'.rown, 

19X’.  ),  reasoning  (e.g.,  Feuorstcin,  19/9;  Hol7iaan,  Glaser,  f.  Pollen) ino,  1976; 
Sternberg  f«  l/eil,  1980),  number  skills  (e.g..  Resnick  &  ford,  in  pie'.<.)_ 
problem  solving  (o.g.,  Siegler,  l'l;;.). 
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To  conclude  this  section,  on  one  view,  information-processing  analyses 
have  been  highly  successful  in  meeting  challenges  that  factorial  methods 
have  met  with  only  partial  success.  But  there  is  at  least  one  other 
point  of  view. 

A  Pessimistic  View 

There  is  another  side  to  the  information-processing  story,  one 
that  may  lead  to  a  more  pessimistic  assessment  of  the  current  state  of  the 
information-processing  paradigm  for  studying  intelligence.  The  pessimis¬ 
tic  view  is  not  inconsistent  with  the  optimistic  one:  It  is  possible 
to  be  both  optimistic  and  pessimistic  simultaneously,  depending  upon  the 
level  of  analysis  one  conducts.  Consider  once  again  the  "four  challenges." 

First,  although  specific  information-processing  theories  arc  usually 
d_i sconfi rmabl e  (see,  e.g.,  Sternberg,  1977,  1980b), the  various  subparadigms 
that  generate  these  theories  are  not  disconfi  nnabl e;  moreover,  they  have 
generally  been  posed  in  ways  that  make  it  very  hard  to  assess  whether 
they  are  succeeding  or  failing.  Consider  some  of  the  major  information- 
processing  subparadigms  currently  being  used,  and  why  I  am  afraid  they 
can  be  "assured"  of  continued  success.^ 

One  subparadigm  is  what  Pellegrino  and  Glaser  (1979)  have  referred  to 
as  the  cognitive-correlates  approach.  In  this  paradigm,  introduc  ■!  by 
Hunt,  Frost,  and  Lunneborg  (1973),  parameters  from  rather  simple  in  forma¬ 
tion-processing  tasks  of  the  kind  used  in  the  cognitive  psychologist's 
laboratory,  e.g.,  the  S.  Sternberg  (1969)  r.c. sory-scanning  task  and  the 
Posner-Mi tr.hel  1  (1  967)  letter-comparison  task,  are  correlated  with  scares 
from  standardized  psychometric  tests  of  mental  ability.  Hunt  et  al. 
computed  these  correlations,  and  found  them  m  nr -rally  to  be  at  thr  level 
of  what  Mischel  (1968)  has  called  "personality  coefficients":  pn.t  of 
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the  correlations  are  at  the  .30  level.  How  does  one  interpret  correlations 
of  this  order  of  magnitude?  Mischel  (19611)  ,  noting  the  fact  that  ,l;,ost 
anything  in  the  personality  literature  correlated  .30  with  anything  else, 
took  a  rather  dim  view  of  the  progress  that  had  been  made  toward  under¬ 
standing  the  nature  of  personality.  He  pointed  to  the  literature  on 
intellectual  abilities  as  a  bright  spot,  however,  because  correlations 
in  that  literature  were  generally  higher.  Out  Hunt  and  his  colleagues 
did  not  share  Mischel's  chagrin,  perhaps  because  researchers  in  the 
field  of  intelligence  have  not  had  to  confront  as  directly  as  have  re¬ 
searchers  in  the  field  of  personality  what  seems  to  be  a  basic  fait-- 
that  within  a  given  domain  such  as  personality  or  ability  nicainre  .pt, 
most  concurrent  test  scores  correlate  at  about  the  .30  level  with  each 
other.  Hunt  and  his  colleagues  were  sanguine,  and  it  is  worthwhile  to 
quote  them  at  length: 

The  general  psychologist  will  have  not'd  that  while  we  have 
shown  significant  results,  we  have  not  shown  strong  correla¬ 
tions.  Indeed,  our  initial  studies  can  be  criticized  on  tin* 
grounds  that  we  have  shown  a  large  nu  bet  of  node  rate  correla¬ 
tions,  rather  than  having  elucidated  any  one  relationship  in 
greater  detail.  Our  data,  then,  may  Ik  more  suggestive  than 
conclusive.  One  of  our  original  points,  however,  was  that 
low  positive  correlations  <n.>  of  inter-,  si.  Although  it  se<  s 
reason. ible  on  a  jiriori  ground',  to  expect  that  the  study  of 
individual  differences  in  cognition  w  .id  locate  the  s.re 
critical  underlying  variable  ■  of  hinan  :  rfot  stance  that  wo  a’1 
be  revealed  by  an  experimental  approach  to  cognition,  this, 
in  fact,  lias  not  happened.  Psychometric.,  and  cognitive  ;>■  ,  - 
choice;;/  appear  to  be  currently  sharing  information  about  -  at, , 
but  by  r.o  means  do  they  chqlicate  tn.  h  other.  At  h  ast,  w. 
see  no  other  way  to  account  for  the  pittern  of  can-  re'ult 
(pp.  IK. -lib) 
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Levels  of  correlations  between  cognitive  tasks  of  the  kind  studied 
by  Hunt  and  his  colleagues  and  psychometric  tests  were  at  similar  levels 
in  subsequent  studies  (see,  e.g.  ,  Hunt,  Lunneborg,  &  Lewis,  1  975),  al¬ 
though  Hunt  (1973)  showed  that  with  much  wider  ranges  of  ability  levels 
(e.g.,  studies  including  retarded  individuals),  it  is  possible  to  boost 
these  correlations,  as  would  be  expected  from  the  increase  in  individual  - 
difference  variation  caused  by  the  expansion  of  range.  The  question 
one  must  pose  regarding  the  "personality  coefficients"  Hunt  lias  obtained 
in  the  majority  of  his  studies  is  whether  there  is  any  set  of  plausible 
results  that  would  differ  from  those  Hunt  obtained.  On  the  one  hand, 
it  seems  unlikely  that  the  relatively  low-level  processing  required  by 
simple  tasks  such  as  comparing  the  physical  appearance  or  names  of  loiter' 
would  yield  high  correlations  with  relatively  high-level  tasks  like 
complex  algebra  word  problems.  Indeed,  a  long-standing  literature  com¬ 
paring  the  approaches  of  Galton  (low-level  tasks)  and  Binet  (high-level 
tasks)  prepared  us  for  the  extreme  unlikelihood  of  such  an  event.  nn 
the  other  hand,  it  seems  unlikely  that  the  cognitive  tasks  and  psyco- 
metric  tests  would  be  wholly  uncorrelated  or  only  trivially  correlated. 
Complex  problem,  such  as  algebra  word  problems  or  verbal  analogies  must 
draw  at  least  to  some  extent  on  the  perceptual  and  memory  prone  ••.os  that 
are  salient  in  the  lower-level  tasks.  Although  these  process'.-,  are  of 
much  less  importance  in  the  higher-level  tasks ,  they  cannot  be  I  . passed 
altogether,  and  to  the  extent  that  tiny  arc  sources  of  indiv  i  du.-.l  dif¬ 
ferences,  they  will  result  in  at  least  some  correlation  betas en  the  cog¬ 
nitive  tasks  and  the  psychometric  tests. 

To  conclude  our  discussion  of  the  i  onni  t  i  vi  •  t  oml  a  t  es  uj,r  sh,  it 


is  not  clear  what  set  of  plausible  outs  on  >s  would  be  i  neons  is  ten*  v.  i  t  h  t> 
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utility  of  the  cognitive-correlates  approach,  at  least  given  Hunt,  Frost, 
and  Lunneborg’s  view  of  what  constitutes  success  of  the  approach:  Correla¬ 
tions  that  can  be  interpreted  as  "moderate"  include  a  very  wide  range  of 
values,  and  moreover,  most  cognitive  measures  are  correlated  at  a  "moderate" 
level  with  each  other.  These  moderate  correlations  tell  us  nothing  about 
directions  of  causal  relationships,  although  one  might  interpret  Hunt  et  al. 
(197b)  as  inferring  from  such  correlations  that  speed  of  access  to  codes  in 
working  memory  is  to  some  extent  causative  of  individual  differences  in  verbal 
ability.  Equally  consistent  with  these  data  would  be  the  possibility  that 
high  verbal  ability  leads  to  quick  access,  or  that  both  high  verbal  ability 
and  quick  access  are  dependent  upon  some  third  variable. 

Suppose  high  rather  than  moderate  correlations  are  obtained  in  a  cognitive 
correlates  study.  What  can  one  then  conclude?  Unless  there  is  a  strong  theory 
underlying  the  newly  discovered  relationship,  probably  not  much.  I  have  seen 
numerous  cases  in  which  the  high  correlations  represent  nothing  more  than  the 
fact  that  the  predictors  and  criterion  or  criteria  are  very  highly  similar  to 
each  other,  sometimes  even  on  their  face. 

A  second  subparadigm  is  what  Pellegrino  and  Glaser  (1979)  have  called 
the  cognitive-components  approach.  In  this  approach,  investigators  analyze 
complex  tasks  such  as  those  actually  found  on  intelligence  tests,  seeking  to 
decompose  performance  on  these  tasks  into  elementary  constituents  of  some  kind, 
usually  component  processes.  In  this  approach,  the  goal  is  to  account  for 
as  much  of  the  variation  between  stimulus  types  as  possible,  usually  using 
either  response  time,  response  choice,  or  error  rate  as  a  dependent  variable. 
Numerous  tests  of  model  fit  can  be  performed  (see  Sternberg,  1?7 !•’•),  so  that  one 
does  not  run  into  the  problem  of  trying  to  decide  what  "moderate"  level  of  fit 
to  be  happy  with.  Ideally,  one  wants  as  high  a  degree  of  model  fit  as  is  pos¬ 
sible,  given  the  reliability  of  the  data. 
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In  this  subparadigm,  discontinuation  of  specific  models  of  information 
processing  is  both  possible  and  likely:  Very  few,  if  any,  models  are  "true" 
in  the  sense  that  they  specify  veridically  just  what  subjects  do.  Moreover, 
it  is  possible  to  reject  alternative  models,  and  to  select  the  best  one  on 
a  tentative  basis.  Although  it  may  not  be  a  "true"  model,  one  can  accept 
it  as  the  best  of  the  available  models  until  a  better  model  is  found.  But 
the  experimental  and  quantitative  rigor  of  the  method  hide  what  I  believe 
to  be  cause  for  at  least  some  concern.  The  approach  in  itself  is  not 
much  more  disconfirmable  than  is  the  cognitive-correlates  approach:  It, 
too,  cannot  fail  if  used  to  full  advantage. 

The  cognitive-components  approach  requires  more  prior  conceptual izat ion 
and  quantitative  sophistication  for  its  use  than  does  the  cognitive-corrc-lates 
approach.  The  investigator  must  go  in  with  a  prior  information-processing  rode! 
that  he  or  she  has  quantified  or  simulated  on  a  computer,  and  can  thus  test  for 
its  validity.  But  if  the  investigator  i_s  able  to  do  these  things,  then  past 
experience  indicates  that  there  is  almost  always  some  componential  model  t1  at  wi 
provide  a  very  good  fit  to  the  data.  Indeed,  there  is  usually  a  linear 
model  that  will  do  so,  since  the  predictions  of  linear  models  usually  accrrd 
well  with  the  predictions  of  ne^.l  i  near  ones.  Thus,  given  sufficient 
cleverness  on  the  part  of  the  investigator  in  manipulating  independent 
variables  that  are  sources  of  solution  difficulty,  and  in  formulating 
a  model  for  how  these  sources  combine  to  yield  the  total  difficulty  of 
the  item  type  being  studied,  past  experience  suggests  that  the  inves¬ 
tigator  is  highly  likely  to  succeed  in  modeling  task  perfornai-.ee.  Of 
course,  these  are  big  "givens,"  and  there  may  be  many  tasks  that  do 
not  seem  susceptible  to  this  kind  of  modeling.  But  there  seem  to  he 
enough  tasks  that  are  susceptible  to  this  kind  of  modeling  to  keep  inves¬ 
tigators  busy  for  quite  a  while.  With  enough  par.-rotors,  of  umi-.c,  sir  r«  • 
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can  bo  guaranteed,  but  previous  modeling  attempts  in  the  literature  sug¬ 
gest  that  most  re  ;arcS«vcan  do  quite  well  even  with  a  fairly  modest  number 
of  parameters.  (•  il.'c‘Aui  f  the  model  cannot  be  rejected  relative  to  the 
true  model,  one  has  no  guarantee  that  the  preferred  model  is 

the  true  model,  since  alternative  models  may  all  predict  a  given  set  of 
data  with  little  or  no  departure  of  observed  from  predicted  values 
{see,  e.g..  Carpenter  &  Just,  1975;  Clark  &  Chase,  1972). 

A  third  subparadigm  is  what  might  be  called  a  training  approach. 

In  this  approach,  one  starts 

with  a  detailed  task  analysis  of  a  cognitive  endeavor  of  par¬ 
ticular  interest  to  the  theorist.  If  the  analysis  is  thorough 
enough,  it  should  be  possible  to  instruct  an  immature  learner 
(or  a  computer)  to  perform  well  on  that  task.  If  the  instruc- 
-  tion  (or  program)  does  not  result  in  an  appropriate  type  or 
level  of  performance,  one  likely  cause  is  that  the  theory  is 
not  specified  in  sufficient  detail.  Ideally,  the  way  in  which 
performance  deviates  from  optimality  will  provide  more  specific 
hints  about  the  ways  in  which  the  theory  needs  elaboration.... 
Finally,  the  point  where  training  ceases  to  be  effective  is  of 
central  importance  to  the  development  of  a  theory  of  intelli¬ 
gence.  The  underlying  assumption  is  that  as  the  difficulty  of 
instructing  some  important  component  increases,  i.e.,  •  e  com¬ 
ponent  begins  to  appear  impervious  to  training,  we  would  argue 
that  the  centrality  of  that  component  to  intelligence  also  in¬ 
creases.  (Campione,  Brown,  &  Ferrara,  in  press) 

This  approach,  which  has  been  used  by  Campione  and  Brown  (197B),  Belmont 

and  Butterfield  (1°71),  and  others,  can  be  helpful  in  telling  us  what 

aspects  of  cognitive  functioning  are  trainable  with  reasonable  amounts 

of  effort,  and  wlut  kinds  of  functioning  resist  training.  It  can  also 

be  helpful  sheer ly  at  a  practical  level  in  improving  people's  cognitive 
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performance.  But  as  an  approach  to  helping  us  find  out  the  nature  of  in¬ 
telligence,  I  am  concerned  that  it,  like  the  other  approaches,  cannot  fail 
Failure  in  training  does  not  really  help  us  much  in  disconfirming  a  theory 
of  intelligence.  Unfortunately,  even  success  in  training  is  at  best  am¬ 
biguous  in  its  interpretation. 

Suppose,  first,  that  one  is  unable  to  train  subjects  in  a  given 
sample  of  a  population  either  to  improve  their  proficiency  in  executing 
a  postulated  component  of  a  theory  of  intelligence,  or  that  one  is  unable 
to  train  them  to  use  the  componentat  all.  What  can  one  conclude?  There 
seem  to  be  at  least  four  highly  plausible  alternative  interpretations  of 
this  outcome.  The  first  is  that  the  component  is  simply  not  a  component 
of  intell igence--that  one  cannot  train  it  because  it  is  not  a  natural 
part  of  a  functioning  cognitive  system.  A  second  interpretation  is  that 
the  component  is  an  aspect  of  intelligence,  but  that  it  is  what  Campione, 
Brown,  and  Ferrara  refer  to  as  "impervious  to  training."  Not  all  intel¬ 
ligent  acts  need  be  accessible  or  even  available  to  consciousness.  For 
example,  part  of  intelligent  functioning  is  the  generation  of  words  to 
represent  one's  thoughts,  although  it  is  difficult  to  imagine  how  one 
could  train  a  skill  like  this  that  is  so  inaccessible  to  consciousness. 

A  third  interpretation  is  that  the  component  is  an  aspect  of  inte'll  ini  nco 
and  is  in  fact  trainable,  but  that  the  training  methods  used  tiro  inade¬ 
quate.  One  can  never  know  for  sure  that  a  failure  to  train  subjects  suc¬ 
cessfully  is  nothing  more  than  a  reflection  of  one's  failure  to  devise 
training  procedures  that  work.  A  fourth- interpretation  is  that  the  com¬ 
ponent  is  an  aspect  of  intelligence  and  that  it.  is  trainable,  (hi*  not  in 
the  population  being  used  in  a  given  study.  For  example,  Campione,  brown. 
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Belmont,  Butterfield,  and  others  have  done  most  of  their  training  work  in 
populations  of  mentally  retarded  subjects.  Could  one  reasonably  conclude 
that  failure  to  train  a  component  process  in  such  a  population  is  an 
indication  that  the  component  is  not  an  aspect  of  intelligence?  One  might 
equally  well  argue  that  its  untrai nabi 1 i ty  in  such  a  population  shows  that 
the  component  is  indeed  an  aspect  of  intelligence.  To  summarize,  the 
interpretation  of  a  failure  in  training  is  quite  equivocal. 

Suppose  instead  that  one  i_s  able  to  train  subjects  in  a  given  sample 
of  a  population  either  to  improve  their  execution  of  a  component  or  to 
execute  the  component  at  all.  What,  then,  can  one  conclude?  Unfortunately, 
not  much.  First,  consider  one  interpretation  of  a  component's  being  "im¬ 
pervious  to  training."  In  the  passage  from  Campione,  Brown,  and  Ferrara 
cited  earlier,  it  is  stated  that  “as  the  difficulty  of  instructing  some 
important  component  increases ,... the  centrality  of  that  component  to  intel¬ 
ligence  also  increases."  But  this  statement  implies  that  the  more  success¬ 
ful  one  is  in  training  a  component  of  intelligence,  the  less  successful  one 
is  in  demonstrating  the  centrality  of  that  component  to  one's  theory  of 
intelligence,  and  vice  versa.  In  other  words,  to  succeed  (in  training)  is 
to  fail  (in  demonstrating  theoretical  importance  of  the  component).  If 
there  was  ever  a  prototypical  example  of  nothing  failing  like  success, 
certainly  this  is  it!  Second,  sometimes  subjects  can  be  taught  to  perform 
tasks  in  ways  that  they  almost  never  would  perform  them  spontaneously; 
other  times,  they  cannot  be  trained  to  perform  the  tasks  in  ways  that  they 


use 
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spontaneously  (Sternberg  &  Weil,  1980).  Would  one  want  to  claim  that  sue-  ( • 

cess  in  training  the  algorithmic  model  but  not  the  mixed  model  indicates 

that  the  mixed  model  is  not  as  good  a  source  of  information  about  the 

nature  of  intelligence,  or  vice  versa?  Certainly  not,  because  in  training 

strategies  for  analogical  reasoning,  the  opposite  pattern  holds:  It  is  quite 

easy  to  train  the  strategy  subjects  use  spontaneously,  but  quite  difficult  ri 

to  train  the  strategies  subjects  do  not  normally  use  (Sternberg  &  Kctron, 

Note  1;  see  also  Sternberg,  Ketron,  &  Powell,  in  press).  One  can  only  conclude  j  j 

that  the  trainability  of  a  set  of  components  (a  strategy)  bears  no  clear  V 

relation  to  the  nature  of  intelligence.  Third,  even  if  one  does  succeed  a 

in  training  a  component  or  strategy,  une  has  no  guarantee  that  the  component  l| 


or  strategy  represents  part  of  what  one  should  call  in  tel  1 i L,ent  performance. 
The  fact  that  a  component  can  (or  cannot)  be  trained  does  not  in  itself 
indicate  whether  that  component  is  a  part  of  intelligence  as  opposed  to,  for 
example,  lower-level  perception.  The  identification  of  the  component  as  one 
aspect  of  intelligence  must  come  from  elsewhere  (prior  theory,  correlations 
with  other  measures,  or  whatever),  which  brings  one  back  to  the  question  that 
one  started  with,  namely,  how  docs  one  identify  the  components  that  constitute 
aspects  of  intelligent  performance? 

I  have  reviewed  three  of  the  major  information-processing  subpurndi gms 
for  understanding  the  nature  of  intelligence,  and  have  concluded  that  in 
at  least  one  respect,  they  face  the  same  difficulty  as  factor  analysis. 

The  difficulty  is  better  disguised,  but  may  be  viewed  as  all  the  more 
pernicious  because  of  its  unobv ioie.ness .  The  problem  is  that  in  at  least  one 
sense,  each  subparadigm  cannot  but  succeed  in  meeting  the  goals  its  users  and 
even  many  of  its  critics  have  set  for  it,  and  hence,  in  at  least 
one  sense,  it  cannot  fail.  I  will  claim  r.cxl  that  information- 
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processing  approaches  share  other  problems  of  factoi  analysis,  although, 
again,  in  an  unobvious  way.  Consider  now  the  "second"  challenge. 

Second,  there  can  be  no  doubt  that  information-processing  approaches 
have  fulfilled  their  promise  of  identifying  component  processes  in  intelli¬ 
gent  performance  (see,  e.g.,  Hunt,  1  973;  Mulholland,  Pellegrino,  f.  Glaser, 
1980;  Sternberg,  1977,  1930c).  But  certain  problems  lurk  beneath  the 
surface:  Our  success  no  longer  seems  as  "successful"  as  we  had  once  hoped 
it  would  seem. 

One  goal  of  information-processing  analysis  was  to  identify  a  unit 
of  analysis  that  would  in  some  sense  be  basic--a  unit,  for  example,  in 
terms  of  which  individual  di  fferr-'ces  in  factor  scores  could  be  under¬ 


stood  (Carroll,  1976;  Sternberg,  1977).  The  hope  was  that  the  component 
process  would  serve  as  such  a  unit,  and  ultimately  provide  a  "common  currency" 
for  the  exchange  of  views  regarding  the  basic  nature  of  intelligence.  But  it 
has  become  increasingly  clear  that  we  really  have  no  way  of  determining 
constitutes  a  basic  unit,  nor  are  we,  I  suspect,  even  clear  as  to  just  what  we 
mean  by  a  basic  unit.  For  example,  we  have  no  way  at  the  present  time  of  knowing 


whether  the  factor  or  the  component  is  the  more  basic  unit.  People  seem  to  take 
all  possible  positions.  Carroll  (  1930  )  now  seems  to  claim  that  the  factor 
is  the  more  basic  unit,  and  that  it  is  responsible  for  individual  differences 
in  components;  I  previously  claimed  that  the  component  was  the  r.ore  basic  unit, 
and  that  it  was  responsible  for  generating  individual  differences  in 
factors  (Sternberg,  1977).  I  now  believe  that  the  question  is  not  a  meaning¬ 
ful  one  in  our  present  state  of  knowledge  (Sternberg,  1930a  ).  We  are 

no  better  able  to  say  which  unit  is  more  basic  than  we  are  a  tile  to  say 
which  case  first,  the  chicken  or  the  egg.  raclor  scores  can  be  regressed 
on  component  scores;  component  scores  can  bo  regressed  on  far '  ->r  ..  'T-'S  . 

What  exp  oriental  or  mnthepiol  ical  operation  would  enable  us  to  rial  wi  ‘ ' 


Wi- 
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any  confidence  that  one  unit  is  more  basic  than  the  other? 

A  second  goal  of  information-processing  analysis  was  to  tell  us  how 
subjects  solve  complex  problems  requiring  intelligent  performance.  At 
one  level,  information-processing  analysis  has  told  us  that.  We  can  say, 
at  least  to  some  order  of  approximation,  that  solution  of  analogies  re¬ 
quires  the  execution  of  operations  such  as  encoding,  inference,  applica¬ 
tion,  and  the  like,  or  that  solution  of  linear  syllogisms  requires  execu¬ 
tion  of  operations  such  as  premise  reading,  processing  of  marked  adjec¬ 
tives,  combination  of  terms  into  a  visualized  spatial  array,  and  the  like 
(see  Sternberg,  1977,  1980b).  But  as  Pellegrino  and  Lyon  (1979),  among 
others,  have  pointed  out,  the  components  identified  in  my  and  others' 
"componential"  analyses  are  black  boxes.  Some  of  the  information  we 
would  be  most  interested  in  would  come  from  our  figuring  out  what  mental 
events  occur  during  encoding,  inference,  premise  reading,  and  so  on. 

In  other  words,  how  does  a  person  do  these  tilings?  One  approach  to  this 
problem  is  to  decompose  information-processing  performance  into  smaller 
components  or  subcomponents  than  the  ones  we  have  used.  Such  an  approach 
at  least  reduces  the  magnitude  of  the  problem.  To  date,  we  have  no  evidence 
that  it  is  capable  of  eliminating  the  problem  altogether. 

A  third  goal  of  information-processing  analysis  was  to  tell  us  what 
intelligent  performance  consists  of.  But  we  need  to  ask  ourselves  whether 
the  processes  we  are  identifying  are  ones  that  we  can  confidently  identify 
as  sources  of  individual  differences  in  interesting  kinds  of  intelligent 
performance  that  occur  in  the  real  world,  such  as  making  a  career  choice. 
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performing  well  in  one's  career,  deciding  how  to  schedule  one's  time  to 

maximize  one's  operating  efficiency,  and  so  on.  I  have  serious  doubts 

that  the  kinds  of  processes  being  identified  in  cognitive-correlate 

analysis  are  on  the  right  track.  Speed  of  naming  two  letters  as  "same" 

or  "different"  does  seem  to  me  quite  removed  from  ordinary  conceptions 

of  intelligent  performance  or  its  antecedents.  I  have  more  confidence 

that  the  kinds  of  processes  being  identified  in  cognitive-component 

analysis  are  on  the  right  track.  For  example,  I  am  prepared  to  believe 

that  “inference"  is  an  integral  part  of  intelligent  functioning  in  the 

real  world.  I  am  much  less  ready  to  believe,  however,  that  "inference" 

of  the  kind  used  in  solving  analogy  test  items  is  the  same  as,  or  closely  related 

to,  say,  inference  in  seeing  relations  between  two  important  historical 

events,  or  between  two  economic  indicators.  We  have  what  I  perceive  to 

be  a  "levels  of  processing"  problem.  There  is  a  large  gap  between  the 

levels  of  inference  used  in  laboratory  or  psychometric  tasks  and  the 

levels  used  in  more  consequential  kinds  of  reasoning  performance.  And  the 

difference  in  levels  may  be  of  a  qualitative  as  well  as  of  a  quantitative 

nature.  Although  there  have  been  reasonably  successful  attempts  to  show 

that  the  parameters  named  in  the  same  way  across  different  laboratory 

information-processing  tasks  are  correspondent  (Chiang  &  Atkinson,  1976; 

Sternberg  &  Gardner, in  press) ;there  have  been  no  attempts  to  relate 
these  parameters  to  performance  in  real-world  performance. 

I  have  claimed  that  although  information-processing  analysis  has 
identified  processes  of  task  performance,  this  identification  has  not  proven 
to  be  the  panacea  many  people  hoped  it  would  be.  Many  of  the  questions 
we  would  like  answered  about  component  processes  still  remain.  And  other 
problems  remain  for  the  information-processing  approach  as  well. 
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Third,  on  the  face  of  things,  the  information-processing  paradigm 
would  certainly  not  seem  to  have  been  exhausted.  But  we  must  ask  ourselves 
exactly  what  it  means  for  a  paradigm  to  be  exhausted.  Is  it,  for  example, 
conventional  factor  analysis  that  has  been  (in  the  opinion  of  some,  at 
least)  exhausted  in  the  factorial  paradigm;  or  is  it  the  types  of  uses 
to  which  we  have  put  conventional  factor  analysis?  I  suspect  that  the 
latter  is  the  case.  We  pretty  much  ran  out  of  tasks  to  factor  analyze 
or  that  we  cared  to  factor  analyze,  and  it  wasn't  clear  where  to  go  from 
where  we  were  with  factor  analysis.  The  critical  question  concerns  not 
so  much  the  technique  itself  as  the  use  to  which  the  technique  is  to  be 
put  to  continue  to  yield  interesting  new  information  about  the  nature  of 
intel 1 i gcnce . 

We  have  not  yet  run  out  of  tasks  upon  which  to  conduct  information¬ 
processing  analyses.  We  have  yet  to  see  (convincing)  process  models  of 
anagram  performance,  remote  associates  performance,  counting  of  cubes 
from  a  three-dimensional  surface  represented  in  two  dimensions,  etc. 

The  question  we  must  ask,  though,  is  that  of  what  is  to  be  gained  from 
isolating  component  processes  from  still  more  tasks  like  the  ones  wo 
have  analyzed?  A  number  of  studies  have  been  done  in  which  component 
process  scores  from  cognitive  tasks  have  been  correlated  with  scores 
from  psychometric  ability  tests  (e.g..  Hunt  ot  al  .  ,  1  973  ,  1  97:>;  Sternberg, 
1977  ,  1 980 (j) .  As  noted  earlier,  the  correlations  between  scores  on  these 
tasks  and  scores  on  psychometric  ability  tests  have  been  less  than  impres¬ 
sive.  But  these  psychometric  tests  have  only  been  proxies  for  the  criteria 
we  really  do  or  at  least  should  care  about --namely ,  performances  in  real- 
world  tasks.  Although  cognitive  process  scores  have  not,  to  my  knew] edm* , 
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been  correlated  with  performance  such  as  school  grades,  supervisory  or 
peer  ratings,  income,  or  whatever  "real-world"  criteria  one  would  like, 

I  am  inclined  to  believe  that  almost  inevitably,  these  correlations  would 
be  lower  than  those  that  have  been  obtained  with  psychometric  tests. 

Such  reduced  correlations  would  stem  not  only  from  the  reduced  reliabili¬ 
ties  of  t ho  criterion  measures,  I  believe,  but  also  from  the  reduced  com¬ 
plexity  of  the  cognitive  components  relative  to  the  composite  psychometric 
tests  that  have  until  now  served  as  the  predictors  of  real-world  per¬ 
formance.  Past  experience  in  research  on  intelligence  has  shown  to  almost 
everyone's  satisfaction  that  higher  predictive  validities  for  complex  out¬ 
comes  are  almost  always  associated  with  greater  complexities  in  predictors. 
Indeed,  increasing  the  reliability  of  a  predictor  by  simplifying  its  struc¬ 
ture  often  results  in  a  decrease  rather  than  an  increase  in  predictive 
validity.  We  must  therefore  ask  ourselves  whether  still  more  information- 
processing  analyses  of  the  kinds  of  tasks  we  have  studied  are  likely  to 
turn  things  around.  I  suspect  they  are  not.  We  need  new  kinds  of  tasks. 

Fourth,  consider  again  the  issue  of  training  component  information 
processes.  There  can  be  little  doubt  that  some  training  of  cognitive 
processes  is  both  possible  and  feasible  (see,  e.g.,  Borkowski  X  Cavanaugh, 
1979;  Feuorstein,  1979;  Holzman,  Glaser,  X  Pellegrino,  1976).  Although 
evidence  supporting  the  durability  and  general i zabi 1 i ty  of  such  training 
is  still  meager,  we  have  cause  to  be  at  least  modestly  optimistic  re¬ 
garding  the  feasibility  of  training  some  cognitive  skills  (Brown  X  Campione, 
in  press).  1  believe  it  important  that  these  training  efforts  continue, 
because  in  terms  of  theoretically-based  programs,  training  of  cognitive 
skills  is  pretty  much  our  best  option  now,  even  if  at  times  these  cognitive 
skills  are  more  narrow  than  ideally  we'd  like,  eventually,  though,  I  think 
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it  important  that  we  supplement  these  training  programs  with  training  in 
real-world  problem  solving  and  decision  making  of  the  kinds  needed  for 
important  events  in  one's  life.  Ultimately,  the  real-world  behaviors 
rather  than  their  proxies  are  what  we  are  interested  in,  and  we  con  at 
least  hope  that  any  training  effects  we  can  get  through  the  proxies  will 
be  strengthened  by  direct  training  of  the  real-world  behaviors  that  we 
hope  to  affect. 

Concl us i ons 

I  have  argued  that  the  present  state  of  research  on  intelligence 
could  be  conceived  of  as  on  the  borderline  of  a  crisis  period:  Conven¬ 
tional  factor-analytic  research  on  intelligence  has  been  less  ..nan  success¬ 
ful  in  meeting  four  challenges  that  confront  intelligence  research;  al¬ 
though  on  the  surface,  information-processing  research  has  been  quite 
a  bit  more  successful  in  meeting  these  challenges,  at  a  deeper  level, 
these  approaches,  too,  have  been  less  successful  than  one  might  wish. 

What  conclusions  can  be  drawn  from  the  review?  I  believe  there  are 
at  least  three  reasonable  ones. 

First,  we  should  be  wary  of  a  trend  in  intelligence  research  to  re¬ 
ject  old  approaches  (or  new  competitive  ones)  to  studying  intelligence 
in  favor  of  our  own  preferred  ones.  In  the  initial  burst  of  out  lur.iasm 
that  accompanies  the  success  of  a  new  methodology,  there  is  an  under¬ 
standable  tendency  to  view  the  new  methodology  as  a  panacea  for  the 
problems  of  old  or  new  competitive  methodologies.  There  is  also  a  tendency 
to  attempt  to  sell  a  new  method  not  only  on  its  virtues,  but  on  the  al¬ 
leged  limitations  of  its  competitors.  Thus,  we  have  been  treated  to 
disquisitions  on  why  cogni ti vo-components  analysis  is  to  be  preferred 
over  factor  analysis  (Sternberg,  1977)  or  cognitive-correlates  aiialy  is 
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(Pellegrino  &  Glaser,  1979);  on  why  analysis  of  covariance  structures 
( Frederi ksen ,  in  press)  or  latent  trait  analysis  (Whitely,  19S0)  is  to 
be  preferred  over  cognitive-components  analysis;  or  on  why,  if  one's 
goal  is  to  isolate  "latent  abilities,"  traditional  factor  analysis  is 
to  be  preferred  over  these  and  other  alternatives  (Carroll,  in  press; 
Guilford,  in  press).  I  am  inclined  to  believe,  however,  that  aJJ  of 
the  methods  now  available  have  overlapping  strengths  and  weaknesses. 

The  best  strategy  to  follow  is  to  attempt  to  show  in  what  respects 
different  methodologies  lead  to  the  same  conclusions  in  some  respects 
and  different  conclusions  in  others  regarding  the  nature  of  intelligence. 

I  also  believe  that  various  methods  in  fact  show  striking  convergences 
in  the  generalizations  to  which  they  lead  us  about  the  nature  of  intelli¬ 
gence  (Sternberg,  1980a,  in  press-b,  in  press-c),  a  belief  I  shall  discuss 
further  shortly. 

Second,  I  think  we  need  to  think  more  about  the  criteria  wo  wish  to 
use  in  evaluating  the  relative  successes  of  various  approaches  to  studying 
intelligence.  One  could  draw  the  conclusion  from  this  review--wrongly , 

I  bel ieve--that  none  of  the  methods  are  very  useful  because  they  are  flawed 
in  so  many  respects.  A  more  valid  conclusion,  I  believe,  would  lie  that 
probably  all  of  the  methods  that  have  been  used  can  lead  to  important  in¬ 
sights  or  to  dull  ones.  Advocates  of  one  approach  can  often  turn  around 
the  arguments  being  leveled  against  their  approach  by  advocates  of  another 
approach  to  apply  to  that  other  approach.  The  value  of  a  contribution 
seems  to  lie  in  how  creatively  and  insightfully  a  given  method  is  used  by 
an  investigator,  rather  than  in  the  method  itself.  What  seems  to  ratter 
most  is  not  what  method  is  used,  but  how  it  is  used.  Attempts  to  argon 
for  or  against  methods,  in  the  abstract,  seem  not  to  bo  terribly  fruitful. 
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A  bettor  use  of  time  might  be  in  arguing  about  ways  in  which  one-  or  more 
methods  might  be  [nit  to  more  productive  use  by  psychologists  interested  in 
using  the  method,  in  effect,  to  think  of  method-investigator  interactions. 

If  no  better  use  can  be  found  for  a  method  at  a  given  time,  it  should  by 
all  means  be  put  into  cold  storage  until,  perhaps  at  some  later  time,  a 
better  use  is  thought  of  for  it. 

Third,  we  need  to  remember  t  ho  oft-repeated  admonition  that  the-  validity 
of  a  theory  can  be  adequately  tested  only  through  the  use  of  converging  opera¬ 
tions  (Garner,  Hake,  &  Erikscn,  1%G).  Any  one  method  for  studying  a  psy¬ 
chological  phenomenon  is  incomplete  in  some  respects,  and  inadequate  in  others. 

But  if  the  same  phenomenon  appears  almost  without  regard  to  the  method  that  is 
used  to  uncover  or  analyze  it,  then  one's  confidence  in  the  validity  of  the 
phenomenon  is  increased.  Although  each  of  the  methods  I  have  reviewed  has  in¬ 
adequacies,  the  use  of  a  combination  of  methods  (including,  of  course,  ones  not 
reviewed  here)  can  provide  a  powerful  demonstration  of  a  phenomenon  of  interest. 

Finally,  I  think  we  need  to  consider  more  carefully  our  choice  of  tasks 
in  studying  intelligent  behavior.  A  common  theme  running  throughout  this  arti¬ 
cle  has  been  that  in  studying  sometimes  remote  proxies  for  interesting  real- 
world  behaviors,  tin  to  lias  been  some  loss  in  1  he  real-world  significance  of  the 
outcomes.  I  would  not  take  the  position  of  Cole  (Note  ?.)  that  the  results  of 
these  studies  have  been  misleading.  Rather,  they  have  been  incomplete.  I  would 
like  to  see  our  laboratory  studies  of  intelligence  supplemented  with  (but  by  no 
means  replaced  by)  studies  of  real-world  behaviors  or  simulations  of  such  behavior 
The  behaviors  studied  should  be  consequential  ones,  such  as  choosing  a  college,  a 
career,  or  a  mate,  or  deciding  whether  or  not  to  pursue  major  surgery.  Such  »•  erf 
has  already  been  undertaken  by  several  investigators.  Fredcriksen  (1'hV,  1W  ; 
Feeder  i  kv  n  ,  Saunders  ,  R  Wand,  I'd.,')  has  been  a  pioneer  in  this  pec  t  in  his 
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direction-sotting  studies  of  the  in-basket  technique  and  in  his  investigations 
of  creative  hypothesis  formation  and  evaluation  in  scientific  thinking  (f'rederik- 
sen  &  Evans,  1974).  Hayes-Roth  and  Hayes-Roth  (1979)  have  proposed  a  theory  of 
real-world  kinds  of  planning,  such  as  the  order  in  which  one  plans  to  carry  out 
a  sequence  of  daily  errands,  and  Goldin  and  Hayes-Roth  (Note  3)  have  found  sys¬ 
tematic  individual  differences  in  the  nature  of  the  planning  process.  Our  c.v:n 
recent  research  lias  begun  to  take  a  more  practical  bent  as  well.  Rick  Wagner 
and  I  are  investigating  the  kinds  of  practical  skills  and  knowledge  of  value 
systems  people  in  everyday  life  and  in  professions  such  as  law  and  psychology 
need  to  get  ahead  in  their  respective  pursuits,  c.g.,  how  people  decide  what 
activities  are  worth  doing  in  limited  amounts  of  time,  and  how  they  budget  their 
time  according  to  the  value  they  place  on  each  activity.  Craig  Smith  and  I  are 
investigating  the  construct  of  social  intelligence,  following  in  the  footsteps 
of  Archer  (1980)  and  Rosenthal,  Hall,  DiMatteo,  Rogers,  and  Archer  (  1  979),  arcr.g 
others,  in  examining  how  people  decode  implicit  communication,  such  as  the  r, cover 
cues  people  emit  in  expressing  approval  or  disapproval.  We  are  interested  in  iso 
lating  components  of  social  intelligence,  if  they  exist,  and  in  relating  them  to 
each  other  and  to  components  of  cognitive  intelligence. 

Laboratory  research  has  been,  and  I  believe,  will  continue  to  be,  useful  in 
isolating  various  aspects  of  intelligent  performance.  A  recent  and  particularly 
promising  development  has  been  in  the  investigation  of  knowledge  represent.;*,  ic^s 
in  people  of  various  levels  of  expertise  who  are  engaged  in  solvin';  compl--  ;  <\  t  1 
such  as  those  found  in  physics  (Larkin,  McDermott,  Simon,  S  Simon,  l°.r;0;  Chi, 
Feltovich,  &  Glaser,  Note  4)  and  geometry  (Gteeno,  1  976).  My  resiarch  and  * 1  si t 
of  others  (e.g.  ,  Mogaboam  &  Pellegrino,  1973)  leads  me  to  believe  that  .  partic¬ 
ularly  promising  route  to  pursue  in  the  study  of  laboratory  tasks  will  !.  t'.'t 
of  what.  I  have  railed  nonent  ronrh<>d ,  i.e.t  novel,  tasks  (Stern!  r;,  in  pi  ■  ). 
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Performance  on  such  tasks  seems  to  be  substantially  more  highly  correlated  with 
performance  on  psychometric  tests  than  performance  on  more  standard  laboratory 
tasks  is  correlated  with  performance  on  the  tests.  I  agree  with  Cole  (Note  3) 
and  Neisser  (1976),  however,  that  v/e  need  to  pay  more  attention  to  macroscopic 
aspects  of  Information  processing  that  are  sometimes  overlooked  in  laboratory 
task  analysis.  At  the  present  time,  our  knowledge  of  high-level  performance 
in  real-world  tasks  is  meager.  But  if  our  goal  in  research  on  intelligence  is 
to  understand  intelligence  as  successful  adaptation  to  and  purposive  action 
In  one's  real-world  environment,  knowledge  about  such  relations  would  seem  to 
be  essential. 
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Preparation  of  this  article  was  supported  by  Contract  N0001  -/i7t?-CC025 
from  the  Office  of  Naval  Research  to  Robert  J.  Sternberg.  I  am  grateful 
to  Janet  Powell  for  comments  on  an  earlier  version  of  the  manuscript. 
Requests  for  reprints  should  be  sent  to  Robert  J.  Sternberg,  Department 
of  Psychology,  Yale  University ,  Box  2 1 A  Yale  Station,  Dew  Haven, 

Connecticut  06520. 

^By  nonconfi rmatory  methods ,  I  mean  those  methods  of  factor  analysis 
that  do  not  use  maximum-likelihood  estimation  procedures  to  test  the 
validity  of  a  prior  structural  model.  Confirmatory  methods  have  cc;;;e  into 
widespread  use  only  during  the  past  decade  or  so  (see,  e.g.  ,  Joreskog,  1  970) 
These  methods,  I  believe,  can  be  highly  useful  in  hypothesis  testing. 

*  o 

The  distinguishability  of  alternative  subparadigms  is  often  hazy, 
especially  since  it  is  possible  to  combine  subparadigms  within  a  single 
study.  Hence,  some  of  the  criticisms  directed  at  use  of  one  particular 
paradigm  may  apply  to  another  in  certain  instances,  whereas  others  of  the 
criticisms  directed  at  that  paradigm  may  not  apply  at  all  in  certain  instant 
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